Challenging the “Law of Diminishing Returns”

"The Law of Diminishing Returns" (Spearman, 1927) states that the size of the average correlation between cognitive tasks tends to be relatively small in high ability groups and relatively high in low ability groups. Studies supporting this finding have tended to contrast very low ability subjects (IQ < 78) with subjects from higher ability ranges and to use tests that have poor discriminatory power among the higher ability levels. In the first study described in this paper, tasks that provide good discrimination among the higher ability levels were used. A sample of High ability (N = 25) and of Low ability (N = 20) 15-years old boys completed four single tests, two with low and two with high g saturations, and two competing tasks formed from these single tests. The results indicated that, contrary to the predictions of the Law of Diminishing Returns, the amount of common variance was greater in the High ability group. It is suggested that the Law of Diminishing Returns does not take into account the factor of task difficulty and that there are situations where the exact reverse of this law holds. A second study again compared correlations obtained with extreme groups (N=28 & N=29), this time on measures of Perceptual Speed, which are easy for all ability levels. Results indicated that correlations among the Perceptual Speed measures were the same for both groups. In neither of these studies was there any support for the Law, which seems to be dependent on the very high correlations obtained from samples at the extreme lower end of the ability continuum. The Law of Diminishing Returns 3 Challenging the “Law of Diminishing Returns” One of the very few "laws" in psychology has been identified in studies of individual differences in cognitive abilities. "The First Law of Intelligence" (see Guttman, 1992) has been known for a long time as the observation of 'positive manifold' i.e., the tendency for cognitive tests to have positive correlations among themselves. As pointed out by Stankov (1983, p. 484), only half of this "law" is of real psychological interest. At the lower levels of intelligence there is a biological limit (i.e., non-living matter), so positive correlation is a necessity. Put simply, this means that if a person’s brain is hardly functioning, then all abilities are affected. Because there is no comparable limit at high-ability levels, the observation of positive manifold among high ability samples is of greater psychological interest. Spearman (1927) also noted that if we divide subjects into groups with respect to their performance on a measure of general cognitive ability, the size of the average correlation tends to be relatively small in high ability groups and relatively high in low ability groups. This finding was a consequence of another law "The Law of Diminishing Returns". Detterman & Daniel (1989) have rediscovered this "law" (cf Deary & Pagliari, 1991; Detterman, 1991). Neither of these authors, however, expressed surprise at the presence of positive manifold within the whole range of ability, including the above-average levels. Instead, they focused on the implications of this finding for the nature of the general factor. For Spearman, this "law" indicates that the more g (or mental energy) "... a person has available already, the less advantage accrues to his ability from further increments in it."(1927, p. 219). Detterman claims that Spearman's interpretation implies that g represents stupidity, not intelligence. According to him, "Each person can be thought of as having a set of independent abilities related to each other by a set of weights specifying each ability's relationship to other abilities in the performance of a particular task or test; g arises from this set of weights in combination with a person's independent abilities" (Detterman, 1991, p. 254). While The First Law of Intelligence is widely accepted, the number of reported studies supporting The Law of Diminishing Returns is much smaller and therefore somewhat less convincing. It appears to have been forgotten until recently, although there have been debates over the years on issues not unrelated to the phenomena described by the Law. For example, there were suggestions by Hoffman (1962, ch.9) that correlations between IQ and academic achievement might be quite invalid for certain types of high ability student. Hoffman directed his criticism at the nature of multiple choice items that were subject to unintended interpretations at the upper end of the ability spectrum, but he might also have introduced the Law to support his argument of diminished correlations among cognitive variables in high ability groups. Similarly, the long-running debate on test bias looked at the question of differential validity for various groups in the population (see Cole & Moss, 1993). Initially, the groups studied were Blacks and Whites but many other groups have also been proposed and studied. The resulting literature shows that in most cases there are no differences between groups in terms of regression slopes for predicting job criteria and in terms of the factorial structure of test batteries within the groups (e.g., Jensen, 1980; Cunningham & Birren, 1980). With the group studies, the focus remained on structural concerns, especially the notion of “invariance”. The work of Linn (1982), The Law of Diminishing Returns 4 who divided the selection bias literature into studies of differences in the predictorcriterion correlation and studies of differences in prediction systems, came closest to discussing issues central to the Law of Diminishing Returns when commenting on varying correlations between predictor and criterion variables for different groups. Now that the Law has again been brought to the attention of the research community there are, in fact, good reasons to question its validity. One set of reasons derives from the fact that proponents of this law have relied upon somewhat obsolete methodology that focuses on correlation coefficients. The usual practice has involved the division of samples into ability groups on the basis of scores on one cognitive test variable and the comparison of correlation coefficients representing relations among other cognitive variables for each of the groups (e.g., Spearman, 1927). A similar logic is used in studies of “invariance”. The literature on measurement invariance is vast (see, for example, Horn & McArdle, 1992; Meredith, 1964; Millsap, 1994). Among other things, the literature on this topic points out that invariance may be present in many parameters of structural equation models, not just in the correlations. Computer programs such as LISREL now provide a means of testing whether trends in correlations may be due to particular features of the structure and allow for the evaluation of these effects on the correlations. It is possible that observed trends in correlations may be the result of some particular statistical features of the data that have nothing in common with the substantive effects of the “Law of Diminishing Returns”. In other words, statistical artefacts have not been eliminated as plausible accounts of trends in correlations. In addition, there would appear to be a weakness in the logic used to support the case for decreasing correlations with high ability groups. We suggest that the metaphor used to explain this law is somewhat limited and that an extension of the metaphor will highlight this deficiency. Thus, in terms of the original metaphor, a weak person will struggle to lift a moderately heavy weight and will benefit considerably from an increase in energy. A strong person will not show much benefit from a similar addition. We do not challenge this logic, nor do we challenge the supporting empirical data gathered from samples of low IQ and average subjects working on standard psychometric tasks. If we extend the metaphor, however, the picture changes. Consider the situation where the weights to be lifted are heavy. A weak person cannot lift the weights at all and does not benefit from a boost of additional energy. A strong person, however, will go close to lifting these weights under normal conditions and will probably benefit considerably from the same energy boost. The Law of Diminishing Returns will not apply under these conditions. In other words, typical intelligence tests are constructed for the “normal” range of abilities and their discriminatory power at the higher ranges may be severely limited because of ceiling effects. If more difficult tests were chosen, these effects would disappear and the lack of discrimination would apply to the low ability subjects. In practical terms, it means that we should be able to find tasks where positive manifold is greater among high ability subjects. One of the aims of the first study was to test this assumption using tasks that are known to impose a heavy cognitive load and which should therefore prove more suited to high ability groups. Competing tasks fall into this category. The Law of Diminishing Returns 5 Study 1: Competing Tasks And The Measurement Of Intelligence in High and Low Ability Groups The term “competing task” is used to describe situations wherein two distinct tasks are competing for attentional resources. These tasks may come through the same or different sensory channels, the stimulus inputs may be simultaneous or interspersed; as long as the individual is forced to do more than one thing at a time, they are what we call competing tasks. Fogarty and Stankov (1982) and Stankov (1983) observed that tasks made a greater demand on general intelligence when measured under conditions where there was competition for limited resources. This meant that one could expect components of these tasks to have higher correlations with measures of general intelligence than their single test counterparts. These same authors later pointed out that despite the greater demand on general intelligence, psychometric indices might not always reflect this demand (Fogarty & Stankov, 1988). The crucial factor is the demand made on general intelligence by the single tests used to form the competing tasks. The metaphor of the human information processing system as a resource limited system (Norman & Bobrow, 1975) can be used to explain the dynamics of the situation. All competing tasks result in greater demand on central resources but if the single tasks themselves place a heavy load on central resources, as indicated by their g loadings, then the system itself has no way of coping with competition between two such tasks. The result may be a partial or complete breakdown in performance. From a psychometric viewpoint, competing tasks can be seen as just another example of a complex task. In fact, they may have some practical utility because of this complexity. Stankov (1993) pointed out that typical tests of intelligence in use today have poor discriminatory power in the above-average range. In terms of the metaphor already employed, using these tests to rank high ability subjects is akin to using a set of moderate weights to determine the rankings in a weight-lifting competition. The true ranking is unlikely to emerge. Stankov suggested that the use of competing tasks with such samples may lead to better discrimination among high ability individuals. One implication of this suggestion is that positive manifold should increase for the high ability group if the tasks used are towards the upper limit of their potential. Under these same conditions, low ability people will fail to cope and positive manifold may not be observed. Such a situation could produce evidence contradicting the Law of Diminishing Returns. In the present study, a group of High ability and a group of Low ability students were selected from a much larger sample and asked to perform four single tests and two competing tasks formed by the coupling of these single tests. The first of these competing tasks combined tests of perceptual processing, the second combined measures of fluid intelligence. The latter task was considerably more complex than the former. According to the Law of Diminishing Returns, the amount of common variance in the battery of tests (i.e., correlations among the tests) should be higher in the Low ability group than in the High ability group. This law has been reported with single tests of abilities, not competing tasks. On the basis of previous work with competing tasks, they were expected to impose processing demands that were likely to exceed the capacity among members of the Low ability group. With the High ability group, on the other hand, task demands were expected to be more in line The Law of Diminishing Returns 6 with available capacity, especially for the easier of the competing tasks. Under these conditions, stronger evidence of positive manifold was expected in the data produced by the High ability group. This is quite at odds with what one would predict on the basis of the Law of Diminishing Returns. Apart from testing the Law of Diminishing Returns, the conditions of the present study also enabled a test of claims arising from earlier studies (Fogarty & Stankov, 1982, 1988; Stankov, 1993) where it was observed that the phenomenon of increased g saturations did not apply when the two single tasks already imposed heavy demands on g, presumably because the demands exceeded available capacity. Under these conditions, g saturations actually decline when the tasks are combined. If this decline is indeed associated with the lack of available cognitive resources, then the decline should be much less marked in a group of high ability subjects. The use of two different ability groups and two different competing tasks one formed from two perceptual tests (low g saturation), the other formed from two fluid intelligence markers (high g saturation) provided an opportunity to explore further the hypothesis that competing tasks increase the demands placed on general intelligence. It was expected that for the Low ability group, g saturations would increase for the variables used in the “perceptual” competing task but decrease for variables in the “fluid intelligence” competing task. In the High ability group, g saturations were expected to increase in both tasks.